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ABSTRACT 

This study investigates differences in instructional and learner factors between two groups of learners 
exposed to online only and blended delivery formats, respectively, in an effort to compare learning 
outcomes and other instructional variables between online and blended delivery methods. Findings 
indicated that no significant differences existed in learning outcomes; however, significant differences 
existed in several instructional and learner factors between the two delivery format groups. Discussions 
about improving online or blended delivery method are presented based upon the research findings. 
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I. INTRODUCTION 

Advances in network and communication technologies have shifted the way we deliver instruction to 
learners in remote locations. Owing to web enhanced communication systems and newer formats of 
media, various innovative instructional methods have provided learning solutions meeting the diverse 
needs of instructors and learners in schools and private organizations. A major concern in adopting the 
new technologies is whether or not educators utilize new technologies for the convenience and efficiency 
in the delivery of educational content [1]. Newer ways to blend traditional instruction with technology 
mediated instructional methods have emerged in an effort to meet the diverse needs of learner satisfaction 
and improve their learning levels. Several research studies claimed the positive effect of blended learning 
for teaching and learning [2], 


While researchers have previously investigated the differences in learning outcomes and other 
instructional conditions between face-to-face instruction and online instruction formats, few studies have 
compared online and blended learning methods that examine differences in learning outcomes or explored 
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mediating mechanisms that may influence learning. More colleges and private sector companies are 
adopting online or blended learning formats for the delivery of their courses and training programs. 
Identifying how the two delivery formats are different in their effectiveness for learners’ learning and 
satisfaction has become an important research topic for instructors and instructional designers to better 
address the teaching and learning issues residing in both delivery formats. 


II. LITERATURE REVIEW 

The origin of online instruction is distance education. Morabito, Sack, and Bhate [3] determined that the 
growth of distance education evolved over four generations: (a) printed instruction, (b) early technology 
in broadcasting systems, (c) online instruction, and (d) web-based teleconferencing. Online instruction is 
defined as any form of learning and/or teaching that takes place via computer network [4]. The 
advancement of online instruction has opened a new era in distance education and contributed to the 
expansion of the educational opportunities by reaching people in various geographical locations thereby 
allowing learners global access to education [5]. Several researchers have advocated the use of online 
instruction for instructional effectiveness and enriched faculty and program development [6], Online 
instruction addresses the issue of time and place constraints on delivering learning experiences to distant 
learners and allows flexible learning modes so students can control their learning path, pace, and 
contingencies of instruction [7]. For private sector organizations, one of the most significant benefits of 
online instruction has been just-in-time delivery of training when employees need learning to effectively 
address performance problems in the workplace [8]. In spite of the many promising features of online 
instruction, certain pitfalls of online instruction have been identified regarding its limited capability to 
engage learners in learning events unless the learners were self-motivated, were active learners [9], and 
possessed strong organizational skills in their learning habits [10], Learners also report the lack of a sense 
of belonging or community during online learning that prevents the development of shared feelings and 
emotions between learners and instructors. Researchers have found that these variables are some of the 
most important factors influencing learner satisfaction and learning transfer effectiveness [11]. Fontaine 
[12] argues that delivering vivid learning experiences to online learners requires creating a sense of 
presence, a feeling of immediacy, and a broad awareness of the real and vivid learning environment. 


Blended instruction has recently received increased usage among academic institutions and private 
companies that have many opportunities associated with time and place [13, 14]. The major thrust of 
blended instruction is to overcome the shortcomings of online instruction and utilize various instructional 
sequencing and delivery strategies to enhance learner satisfaction while also achieving increased learning 
outcomes. Among the many definitions available, three representative definitions of blended instruction 
include: (a) a learning method with more than one delivery mode is being used to optimize learning 
outcomes and reduced cost associated with program delivery [15], (b) any mix of instructor-led training 
methods with technology-based learning [2], and (c) the mix of traditional and interactive-rich forms of 
classroom training with any of the innovative technologies such as multimedia, CD-ROM, video 
streaming, virtual classroom, email/conference calls, and online animation/video streaming technology 
[16]- 


Singh and Reed [15] have proposed six combinations of blended instruction regarding specific patterns of 
blended learning typologies: (a) offline and online learning, (b) self-paced, live, and collaborative 
learning, (c) structured and unstructured learning, (d) custom content with off-the-shelf content, (e) work 
and learning, and (f) ingredients blending synchronous physical formats, synchronous online formats, and 
self-paced, asynchronous formats. Reasons for using blended instruction include: improved pedagogy, 
easy access to knowledge, more interaction among learners, personal presence, cost effectiveness, and 
ease of revision of learning content [17]. 
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The extensive review of related literature about online and blended instruction validated the usefulness 
and effectiveness of each learning delivery format in relation to learner satisfaction and learning 
outcomes. Few studies, however, have empirically tested how the learners in each delivery format are 
different in terms of learning, instructional satisfaction, and learning involvement and motivation. 
Additionally, few studies have been conducted to identify the differences in learners’ learning application 
between online and blended learning environment. Here, the term ‘application of learning’ refers to the 
degree to which learners use and apply learned knowledge and skills to their current studies or to current 
jobs and tasks. Considering the compelling need to identify evidence of learning effectiveness in both 
public and private sector organizations, evaluating learning application outcomes becomes a critical issue 
for researchers in an educational discipline. 


III. PURPOSE AND METHODOLOGY 


A. Purpose 

The purpose of this research study was to identify the differences in instructional and learner factors, 
students’ learning, and application of learning between two groups of undergraduate students who took a 
program evaluation course through an online only or blended delivery format at a southeastern university. 
The research questions for this study asked: 

Do learners in online and blended delivery format show significant differences in learning and learning 
application before and after the course? 

What are the perceived differences in instructional satisfaction, learning, and application of learning 
between the learners in blended and online delivery format? 

What are the reasons facilitating or inhibiting the learners’ learning and learning application in blended 
and online delivery group? 


B. Sample 

A group of undergraduate students was asked to participate in this study to assess learning outcomes 
based on the learner and instructional variables. The subjects for the study included 125 students (39 male 
and 86 female) who took a program evaluation course at the University of Tennessee. Most of the 
students were majoring in Human Resource Development at the university. Among the 125 students, 59 
students took the course through online delivery format and 69 through blended delivery format using 
classroom and online instruction. Regarding the students’ age, 87 (67%) students were between 18-19, 27 
(21%) students were between 20-29, 6 (5%) students were between 30-39, and 5 (4%) students were 40 
or above. For the students’ distance learning experience, 99 (80%) students replied they had taken at least 
one distance learning course prior to the course. Regarding employment status, 30 students were fulltime 
students, 59 students had part-time jobs, and 36 students had fulltime jobs. 


C. Instrument and Procedure 

The study utilized a multi-method approach that combined closed-ended and open-ended questions in an 
online questionnaire. Linking quantitative and qualitative data in this way enabled confirmation and 
corroboration through triangulation, provided richer detail, and helped to initiate new lines of thinking 
[18, 19]. The questionnaire was developed to obtain the learners’ perceived degree of learning, learning 
application, and instructional quality of the course. The questions for both the open-ended and closed- 
ended parts of the questionnaire were written in a language that was familiar to the learners using 
terminology taught in the course. 
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The closed-ended part of the questionnaire used a five point Likert-type scale to measure the perceived 
degree of learning (1 “do not understand” to 5 “completely understand”) and the perceived degree of 
learning application (1 “none” to 5 “frequently use”) for the eighteen learning objectives of the course 
taught throughout the semester. The study utilized a test set to assess actual learning gain before and after 
each semester. Overall, a reliability alpha was .95 for the learning, .93 for the learning application, and 
.70 for the test respectively. To collect the pre- and post- survey data, the students were asked to 
participate in the surveys conducted online at the beginning and at the end of each semester. The data 
collection was conducted for 8 semesters between 2001 and 2005. 


The open-ended part of the questionnaire asked questions about the reasons for high or low perceived 
learning and learning application if each learner responded their perceived learning and learning 
application of any learning objective at 1 or 2 in the rating scale. The open-ended part of the questionnaire 
followed the closed-ended part of the questionnaire. The open-ended part of the questionnaire also asked 
the learners’ satisfaction with instructional factors such as instructor, learning activities, group work, 
learning support, and suggestions to improve the course. The researchers believed the open-ended 
questions in the survey strengthened the study by investigating in-depth information of the learners’ 
insight about the course delivery. 


D. Data Analysis 

1. Quantitative Analysis 

Basic descriptive statistics were used to analyze the test scores and the perceived degree of learning, 
application of learning, and instructional quality responded by the learners. Paired t-tests were used to 
compare population mean scores for the learning gain before and after the course. We also used ANOVA 
to assess the differences in dependent variables between the comparison groups. 


2. Qualitative Analysis 

The investigators conducted domain analyses employing content analysis procedures [20]. These analyses 
involved sorting through the open-ended responses and identifying themes and patterns that characterized 
the reasons that promoted or hindered the learners’ learning and application for the two different delivery 
formats. After content analysis, cumulative frequencies and percentages for similar types and attributes 
identified in the domain categories were calculated to determine how often similar types were elicited. 
This allowed the investigators to include those terms elicited most frequently and to gain a better 
understanding about the distribution of beliefs across domain categories. Linking qualitative and 
quantitative data in this manner helps investigators see the trends in the data more easily and rapidly by 
looking at distributions [19]. Domain categories and tentative assertions were reviewed by study 
participants who gave feedback. Peer researchers examined the tentative assertions as well and gave 
constructive comments. Conducting member checks and peer examination in this manner helped the 
researchers enhance authenticity and trustworthiness of the findings [21]. 


E. Context of the Course 

The course was developed to teach curriculum content on learner and program evaluation for HRD 
undergraduate students. The course was delivered totally online for two years, and then, through a 
blended delivery format for another two years utilizing classroom instruction and online delivery 
methods. Regarding the online learner group, the instructor developed thirteen online learning modules 
and the workload of one module was equivalent to that of one week’s classroom instruction. Four sub- 
learning sections comprised one learning module. Learning modules provided subject content in learner 
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and program evaluation and various types of media such as texts, graphics, tables, audio, and video clips 
were used to effectively deliver the learning content to the online learners. Several interactive learning 
activities including online discussions, case study analyses, and online tests and surveys were utilized 
within the modules to provide the learners with opportunities to apply learned content during learning. All 
learners were asked to attend the first and last class meeting for course orientation and group project 
presentation respectively. All learners were also asked to complete individual projects to apply learned 
content; projects can later be used as personal portfolios in program evaluation for future job searches. 


Regarding the blended learner group, half of the instruction was conducted in class and half was delivered 
through online delivery. Learners were required to attend weekly classroom instruction in which the 
instructor provided presentation on each week’s major course content. After each week’s classroom 
instruction, the learners were required to complete online learning modules to reinforce their classroom 
learning. The online learning module of the course included various learning activities such as review of 
more related learning content, links to learning resources, group discussions, and application of learning 
content through assignments and group and individual projects. 


IV. FINDINGS 

A. Differences in Learning and Perceived Learning Application 

The results indicated that learners experienced a significant increase in perceived and actual learning. 
Further, both online and blended learning groups reported a significant increase in their perceived and 
actual learning (see Table 1). An ANOVA was also conducted to assess differences in the learners’ 
perceived and actual learning, perceived learning retention, and perceived learning application with mean 
scores for the different delivery groups. The results indicated that delivery format groups did not reveal 
any differences for the dependent variables in course outcomes. 


Table 1. Difference in Learning Before and After the Course Based on Delivery Format 


Delivery 

Format 

N 

Pre/Post 
Perceived 
Learning Mean 
(SD) 

Effect 

Size 

Sig. 

N 

Pre/Post Test 

Mean (SD) 

Effect 

Size 

Sig. 

Online 

59 

3.01 (.65) 

3.72 (.58) 

.487 

<.001 

55 

8.14 (2.03) 

11.20 (3.48) 

.504 

<.001 

Blended 

66 

3.14 (.65) 

3.84 (.52) 

.503 

<.001 

66 

8.35 (2.27) 

11.14(3.60) 

.408 

<.001 

All 

125 

3.08 (.65) 

3.78 (.55) 

.495 

<.001 

120 

8.26 (2.13) 

11.26 (3.55) 

.449 

<.001 


B. Differences in Instructional and Learner Variables 

Some meaningful differences occurred regarding the differences in instructional and learner factors based 
on the two delivery formats. First, learners in the online delivery format had a significantly higher mean 
score for instructional difficulty level than those in blended delivery format. Second, learners in online 
delivery format experienced significantly higher workload for their study than those in blended delivery 
format. In contrast, learners in online delivery format felt significantly less learning support during study 
than their counterpart learners in blended delivery format. Table 2 presents the mean scores, standard 
error scores, and p values of the instructional and learner factors. 
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Table 2. Differences in Instructional and Learner Factors Based on Delivery Format 


Variables 

Delivery 




Effect 

ANOVA 

N 

Mean 

SD 

Size 

p value 


Format 

Difficulty 

Online 

58 

3.57 

.82 

.063 

.005 

Level 

Blended 

66 

3.09 

1.02 

Study 

Online 

58 

3.95 

.62 

.150 

<.001 

Workload 

Blended 

66 

3.32 

.86 



Learning 

Online 

59 

3.37 

.99 

.084 

.001 

Support 

Blended 

66 

3.89 

.74 


C. Reasons for High or Low Learning and Application 

Learner responses to the survey questions identified the most influential reasons supporting and hindering 
learning and application. The reasons given for instructional effectiveness were identified as the most 
influential factors for learning (online 49.5%, blended 52.4%, all 51%). From the various reasons under 
the instructional effectiveness category, “clear and concise learning content” was found to be the most 
important reason for high learning for both groups while “review and repetition of learning” was 
identified more frequently by the blended learner group (17 responses) than the online learner group (4 
responses). 

Regarding the reasons for low perceived learning, instructional ineffectiveness was also found to be the 
most important category negatively influencing the learners’ learning as a whole. Under the instructional 
ineffectiveness category, some specific reasons quoted by the learners are: “I really didn't completely 
understand these,” “I need more examples to comprehend these concepts,” and “Lesson was too complex 
for me to understand.” Other important categories that appeared to negatively influence learning include 
“lack of understanding,” “lack of interest,” and “not related to my work.” Tables 3 and 4 report the 
verbatim categories given for high and low perceived learning along with the frequencies and percentages 
for these reasons. 


Table 3. Reasons for High Learning by Delivery Format 


Reason Category 

Online (%) 

Blended (%) 

All (%) 

Instructional effectiveness 

47 (49.5) 

66 (52.4) 

113 (51.1) 

Previous learning 

15 (15.8) 

18(14.3) 

33 (14.9) 

Related to my current or future jobs 

12 (12.6) 

15 (11.9) 

27 (12.2) 

High interests in the learning content 

8 (8.4) 

10(7.9) 

18 (8.1) 

Opportunity to practice learning 

6 (6.3) 

6 (4.8) 

12 (5.4) 

Personal learning effectiveness 

5 (5.3) 

6 (4.8) 

11 (5.0) 

Personal motivation for learning 

2(2.1) 

5 (4.0) 

7 (3.2) 

Total 

95 

126 

221 

Table 4. Reasons for Low Learning by Delivery Format 

Reason Category 

Online (%) 

Blended (%) 

All (%) 

Instructional ineffectiveness 

22 (39.3) 

19 (46.3) 

41 (42.3) 
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Lack of understanding 

Lack of interest in the learning content 

Not related to my work 

Lack of personal effort 

Personal dislike of online instruction method 

Lack of opportunity to use learning 

Interruptions during learning 

Total 


13 (23.2) 

6(14.6) 

19(19.6) 

11 (19.6) 

5 (12.2) 

16(16.5) 

4(7.1) 

5 (12.2) 

9 (9.3) 

2 (3.6) 

3 (7.3) 

5 (5.2) 

3(8.1) 

0 (0.0) 

3(3.1) 

0 (0.0) 

3 (7.3) 

3(3.1) 

1 (1.8) 

0 (0.0) 

1 (1.0) 

56 

41 

97 


Some categories seemed to influence the learner’s perceived application of learning positively or 
negatively. Verbatim responses listed in Tables 5 and 6 indicate that the most important categories 
positively influencing learners’ perceived learning application include “opportunity to use learning,” 
“applicable to my work,” and “personal interest to use learning.” The most important categories 
negatively influencing learning application were identified as “lack of understanding,” “not related to my 
job,” “not enough opportunity to use during class,” “lack of opportunity to use in my job,” and “lack of 
motivation to apply.” One interesting finding between the two comparison groups is the difference in two 
specific responses in “opportunity to use learning” for high perceived learning application. Learners in the 
blended delivery group responded more frequently to “use learning during class activities, assignments, 
and for other classes or personal situations” than the learners in online delivery group. 


Table 5. Reasons for High Application by Delivery Format 


Reason Category 

Online (%) 

Blended (%) 

All (%) 

Opportunity to use learning 

37 (49.3) 

59 (56.2) 

96 (53.3) 

- To use in learning activities and assignments 

19 

38 

57 

- To use for other classes and personal situations 

6 

11 

17 

- To use in my job 

8 

8 

16 

- To review learning through quizzes 

4 

2 

6 

Applicable learning content to my work or career 

18 (24.0) 

17(16.2) 

35 (19.4) 

Personal interest 

9(12.0) 

15 (14.3) 

24(13.3) 

Experience from previous learning 

4 (5.3) 

5 (4.8) 

9 (5.0) 

Personal interest 

3 (4.0) 

4 (3.8) 

7 (3.9) 

Personal motivation to apply 

3 (4.0) 

3 (2.9) 

6 (3.3) 

Because of repetition and emphasis of information 

1(1.3) 

2(1.9) 

3 (1.7) 

Total 

75 

105 

180 

Table 6. Reasons for Low Application by Delivery Format 

Reason Category 

Online (%) 

Blended (%) 

All (%) 

Lack of understanding of learning content 

12 (28.6) 

10 (28.6) 

22 (28.6) 

Not related or applicable to my job 

9(21.4) 

4 (11.4) 

13 (16.9) 

Not enough opportunity to use during class 

7(16.7) 

5 (14.3) 

12(15.6) 

Lack of opportunity to use learning in my job 

4 (9.5) 

6(17.1) 

10(13.0) 
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Lack of motivation to apply 
Learning activities were not related 
Lack of clear instruction for application 
Not stressed to apply 

Too much content to apply for a given time 
Total 


5(11.9) 

4 (11.4) 

9(11.7) 

2 (4.8) 

1 (2.9) 

3 (3.9) 

1 (2.4) 

2 (5.7) 

3 (3.9) 

0 (0.0) 

3 (8.6) 

3 (3.9) 

2 (4.8) 

0 (0.0) 

2 (2.6) 

42 

35 

77 


Learners in both groups indicated similar responses regarding the instructional activities perceived as 
helpful for learning. Among all instructional activities used for the course, learners in both groups 
perceived group/individual projects as the most important learning activity followed by discussion 
activities, class assignments, review/pre/post tests, case studies, multimedia cases and scenarios, lecture, 
and online learning modules. Table 7 reports the verbatim responses and the frequency and percentage of 
each learning activity for both learner groups. 


Table 7. Helpful Instructional Activities by Delivery Format 



Online (%) 

Blended (%) 

Total (%) 

Group or individual project 

29 (25.4) 

47 (28.5) 

76 (27.2) 

Learning and discussion activities 

20(17.5) 

27 (16.4) 

47(16.8) 

Class assignments 

20(17.5) 

21 (12.7) 

41 (14.7) 

Review and pre/post tests 

14(12.3) 

23 (13.9) 

37(13.3) 

Case studies 

9 (7.9) 

10(6.1) 

19(6.8) 

Multimedia cases and scenarios 

10(8.8) 

8 (4.8) 

18 (6.5) 

Lecture 

4 (3.5) 

9 (5.5) 

13 (4.7) 

Online learning modules 

2(1.8) 

1 (6.1) 

12 (4.3) 

Examples 

0 (0.0) 

4 (2.4) 

4(1.4) 

Instructional feedbacks 

1 (0.9) 

3(1.8) 

4(1.4) 

Online chats 

3 (2.6) 

0 (0.0) 

3(1.1) 

Reading materials 

1 (0.9) 

2(1.2) 

3(1.1) 

Peer reviews 

1 (0.9) 

1 (0.6) 

2 (0.7) 

Total 

114 

165 

279 


V. DISCUSSION AND IMPLICATIONS 
A. Effect of Delivery Format on Learning and Application 

Data analysis revealed that the two learner groups in online and blended delivery formats did not show 
any significant differences in the mean scores for perceived and actual learning and perceived and actual 
learning retention, while all learners, regardless of the delivery options, indicated a significant increase in 
perceived and actual learning before and after the course. This finding resembles similar findings from 
previous studies that compared learning outcomes of traditional classroom with those of distance 
education [22, 23, 24, 25], Also, the study findings replicate similar findings that have suggested there is 
no significant difference in learning application between the traditional classroom instruction and distance 
education delivery format [22, 26]. From this finding, the researchers could conclude that instructional 
delivery format may not affect learners’ learning or application of learning to a significant degree. 
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B. Instructional Factors and Conditions Affecting Learning and Application 

The study revealed several meaningful findings when the researchers conducted further analyses to detect 
the differences that may exist between the two delivery formats. First, the learners in online delivery 
group reported more workload than those in blended delivery group while they claimed less learning 
support than the learners in blended delivery group during their learning. This finding draws some 
implications for the importance of the learners’ psychological state in blended and online learning 
environment especially since learners may feel unsupported and experience an increased workload if they 
lack the sense of presence or belonging [27]. From a practical perspective, the findings suggest that an 
important consideration in designing online or blended instruction is to include instructional activities and 
collaboration opportunities that enhance the learners’ emotional engagement with peers and instructors. 
The present study recommends several instructional strategies to satisfy learners’ sense of presence and 
belonging during online or blended learning including: (a) providing immediate feedback on learners’ 
questions and timely technical support; (b) asking short questions checking the understanding of major 
learning content at frequent intervals during instruction; (c) sending learners’ learning progress report on 
a regular base to promote learners’ motivation for learning achievement; and (d) using humor so the 
learners feel emotionally refreshed and engaged [22]. 


Second, the blended delivery format seems to provide clearer instructions to learners rather than using the 
online delivery format alone. In this study, the online learners experienced more challenges and obstacles 
in achieving similar learning levels than the learners in blended delivery group. They also provided 
relatively more claims for their lack of understanding as their reasons for low learning than the blended 
delivery learner group. This finding suggests that blended delivery methods may provide clearer and 
learner centered instructions than online only delivery method. The issue of instructional clarity in 
delivering online instruction has been reported from other studies [22, 26]. To resolve this issue, 
instructors or instructional designers of online instruction are advised to adjust the level of instructional 
content and apply instructional variation to meet the varying learning levels of online learners. 


Certain instructional activities were considered more important than others in this study regarding 
instructional activities and methods used for both delivery formats. For example, group and individual 
projects, discussion activities, and class assignments were noted as the most effective learning activities 
for the learners’ learning as a whole. This finding has implications for the importance of learning 
application for greater learner satisfaction and increased learning regardless of the different instructional 
formats. That is, learners seemed to value those learning activities that they could apply learned 
knowledge and skills to personal situations more than merely understanding instructed learning content in 
both online and blended learning environment. To promote more application of learning during blended 
or online instruction several instructional strategies are deemed effective. Those strategies include: (a) 
making learning content “applicable” as suggested by Baldwin and Ford [28]; (b) utilizing reflective 
activities that assist learners to apply their learning to personal situations during instruction [29]; and (c) 
embedding a structured learning process composed of a step-by-step guided practice after a segment of 
instruction, followed-up by individual practice to reinforce the guided practice, and independent practice 
through class assignments to master learned content in different application context [22]. 


VI. FUTURE STUDY AND LIMITATIONS 

This study contributes several meaningful findings to the educational knowledge base regarding how to 
design blended or online instruction to better address learners’ satisfaction and learning outcomes. 
Furthermore, this study sought to address the issues of learning application in online and blended learning 
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environment, which is directly connected to the issue of learning effectiveness in today’s organizations. 
As noted from other research, the nature of the open-ended questions in a survey format may not produce 
the rich, thick description that is characteristic of one-on-one interviews; however, including open-ended 
questions helped strengthen the study by delving further into student insight. 


Even though this study revealed several meaningful research findings, the study findings are limited to 
online and blended learning environments occurring in college settings. To further generalize these 
findings, future studies using a broader population including private sector organizations are strongly 
recommended. Regarding the issues related to what constitutes blended instruction, this study utilized one 
type of blending mix of instructional method using classroom and online instruction. For generalization, 
future studies are needed to verify how other types of instructional blending influence learning and 
learning application. 
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IX. APPENDIX: QUESTIONNAIRE 


Course Exit Survey 

The purpose of this questionnaire is to assess students’ competency level and instructional quality of this 
course. Your participation in this study is voluntary. The data will be stored securely and will be used for 
the research purpose only. It is expected to take less than 15 minutes to complete this survey. Submission 
of this form constitutes your consent to participate. Please answer the following questions to the best of 
your knowledge. 

Gender: Male ( ) Female ( ) 
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Student Learning and Application 

The following question items will assess your learning and application of the course objectives. For the 
application items, rate your perceived degree of application of the learning through the class activities, 
such as learning activities, assignments, quizzes, group project, individual project, and case studies. Or 
you can refer the application to your personal occasions, such as application to other classes, jobs, tasks, 
or personal lives. 


Learning Scale 

For each of the following items, I . 

1 2 3 4 5 

don't understand at all < > completely understand 


Application Scale 

During the class, I could the following items to certain situations. 

1 2 3 4 5 

not apply at all < > frequently apply 


1. Describe the difference between performance and competency. L: 1 — 2 — 3 — 4 — 5 

A; 1 — 2 — 3 — 4 — 5 

2. Describe the general process of performance management. L: 1 — 2 — 3 — 4 — 5 

A: 1 — 2 — 3 — 4 — 5 

3. Determine appropriate steps to conduct performance L: 1 — 2 — 3 — 4 — 5 

measurement. A: 1 2 3 4 5 

4. Develop performance objectives. L: 1 — 2 — 3 — 4 — 5 

A; 1 — 2 — 3 — 4 — 5 

5. Develop the table of specification to generate instructional L: 1 — 2 — 3 — 4 — 5 

objectives. A: 1 — 2 — 3 — 4 — 5 

6. Develop performance measurement instruments, such as test, L: 1 — 2 — 3 — 4 — 5 

survey, interview and observation guide. A- 1 2 3 4 5 

7. Evaluate performance measurement instruments to check L: 1 — 2 — 3 — 4 — 5 

validity and reliability. A- 1 2 3 4 5 

8. Describe when to use basic statistics such as mean, median, L: 1 — 2 — 3 — 4 — 5 

mode, standard deviation, and t-test. A- 1 2 3 4 5 

9. Differentiate training from non-training needs. L: 1 — 2 — 3 — 4 — 5 

A; 1 — 2 — 3 — 4 — 5 

10. Clarify the difference between evaluation and research. L: 1 — 2 — 3 — 4 — 5 

A: 1 — 2 — 3 — 4 — 5 

11. Describe the importance of evaluation for program L: 1 — 2 — 3 — 4 — 5 
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enhancement and performance improvement. A: 

12. Describe when and why to use su mm ative and formative L: 

evaluation. a- 

13. Identify key steps to conduct an evaluation. L: 

A: 

14. Develop an evaluation plan. L: 

A: 

15. Describe major categories of the evaluation standards. L: 

A: 

16. Describe when to use each level of the four-level evaluations L: 

proposed by Kirkpatrick. A: 

17. Describe barriers and success factors of training transfer inL: 

organizations. A - 

18. Describe major components of return on investment (ROI) L: 

evaluation. A- 


1 — 2 — 3 — 4 — 5 
1 — 2— -3 — 4 — 5 
1 — 2 — 3 — 4 — 5 
1— -2 — 3— -4— -5 
1 — 2— -3 — 4 — 5 

1— -2 — 3 — 4 — 5 
1 — 2 — 3 — 4 — 5 
1 — 2 — 3 — 4— -5 
1 — 2 — 3 — 4 — 5 
1 — 2 — 3 — 4 — 5 
1 — 2— -3 — 4 — 5 
1 — 2— -3— -4 — 5 
1 — 2 — 3 — 4 — 5 
1— -2 — 3 — 4 — 5 
1 — 2 — 3 — 4 — 5 


To answer questions 19 through 22, take a quick review of your responses from question 1 through 
18 in this section. 

19. What are the reasons for high learning if you checked any question item from 1 through 18 at either 4 
or 5 of the learning scale? 

Reason 1 : 

Reason 2: 

Reason 3: 

20. What are the reasons for low learning if you checked any question item from 1 through 18 at either 1 
or 2 of the learning scale? 

Reason 1 : 

Reason 2: 

Reason 3: 

21. What are the reasons for high application if you checked any question item from 1 through 18 at 
either 4 or 5 of the application scale? 
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Reason 1 : 

Reason 2: 

Reason 3: 

22. What are the reasons for low application if you checked any question item from 1 through 18 at 
either 1 or 2 of the application scale? 

Reason 1 : 

Reason 2: 

Reason 3: 

Instructional Design Factors 

The following question items will assess the quality of the instructional design of the HRD455 course. 
Please answer each question according to the illustrated scale. 

Course Outcomes and Student Effort/Involvement 

For the following question items 1 through 8, rate your perception about the course 
quality compared to other courses. 

1 2 3 4 5 

Much less Less About the same More Much more 


1 . My learning increased in this course. 

2. 1 made progress toward achieving course objectives. 

3. My interest in the subject area has increased. 

4. This course helped me to think independently about the subject matter. 

5. This course actively involved me in what I was learning. 

6. 1 studied and put effort into this course. 

7. 1 was prepared for each class (such as reading assignments). 

8. 1 was challenged by this course. 


1 — 2 — 3 — 4 — 5 
1 — 2— -3— -4 — 5 
1 — 2 — 3 — 4 — 5 
1— -2 — 3 — 4 — 5 
1 — 2 — 3— -4 — 5 
1 — 2 — 3 — 4 — 5 
1 — 2 — 3 — 4 — 5 
1 — 2 — 3 — 4 — 5 
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Faculty and Student Interaction (9-12) / Instructional Quality (13-18) / Exams (19-22) 


For the following question items 9 through 18, rate your perception about the course 
quality according to the following scale. 

1 2 3 4 5 


Ineffective 


Somewhat 

effective 


Moderately 

effective 


Effective 


Very 

effective 


9. The instructor's helpfulness and responsiveness to students 

10. The instructor's concern for student progress 

1 1 . The availability of extra help for this class 

12. The instructor's willingness to listen to student questions and opinions 

13. The instructor's use of examples or illustrations during the instruction 

14. The instructor's use of challenging questions during the instruction 

15. The instructor's command of the subject matter 

16. The instructor's ability to make clear and understandable presentations 
of information 

17. The instructor's way of summarizing or emphasizing important points 
during instruction 

18. The instructor's use of web technologies as aids in instruction 

19. The information about how you would be graded 

20. The clarity of quiz questions 

21. The quiz's coverage of important aspects of that week's class 

22. The helpfulness of assignments and class projects in understanding the 
main content of the class 


1 — 2 — 3 — 4 — 5 
1 — 2—3 — 4—5 
1 — 2 — 3 — 4 — 5 
1 — 2 — 3 — 4 — 5 
1 — 2 — 3 — 4 — 5 
1 — 2 — 3 — 4 — 5 
1 — 2 — 3—4 — 5 
1 — 2 — 3 — 4 — 5 

1 — 2 — 3 — 4 — 5 

1— -2 — 3—4 — 5 
1— -2 — 3 — 4 — 5 
1 — 2 — 3 — 4—5 
1 — 2 — 3—4 — 5 
1 — 2—3 — 4 — 5 


23. The level of difficulty of this course 


INot difficult 


2 

Somewhat difficult 


3 

Moderately difficult 


4 5 

Difficult Very difficult 


Work Load 

24. The general work load for this course in relation to other courses of equal credit 

1 2 3 4 5 6 

Much lighter Lighter About the same Heavier Much heavier N/A 

*N/A - Not applicable 

25. The length of the study time for this course in relation to other courses of equal credit 


41 


Online vs. Blended Learning: Differences in Instructional Outcomes and Learner Satisfaction 


1 2 3 4 5 6 

Much shorter Shorter About the same Longer Much longer N/A 

26. The assignments for this course in relation to other online courses of equal credit 

1 2 3 4 5 6 

Much lighter Lighter About the same Heavier Much heavier N/A 

27. The assignments for this course in relation to other classroom courses of equal credit 

1 2 3 4 5 

Much lighter Lighter About the same Heavier Much heavier 

28. What is your average study time for one learning module (in minutes)? 


Student Support 

29. How effective was the student support for the following items? 

Graduate Teaching Assistant 

12 3 4 

Ineffective < > 

Feedback for Learning Activities 

1 2 3 

Ineffective < 


Responsiveness to Study Questions 

1 2 

3 

4 

5 

Not fast < 


> 

Very fast 

Technical Support 

1 2 

3 

4 

5 

Ineffective < 


> 

Very effective 


4 5 

-> Very effective 


5 

Very effective 
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